1. Field of the Invention
The present invention relates to a method, system and computer program product for checkpointing. More particularly, the present invention relates to a method, system and computer program product of checkpointing model building for data mining.
2. Description of the Prior Art
The present invention relates to a method, system and computer program product of checkpointing model building for data mining. Generally, software that performs compute intensive tasks, such as data mining, implement long running model build data analysis algorithms. These algorithms may terminate abnormally or fail before their completion resulting in the loss of time, waste of computing effort and inability to make time critical decisions. Algorithms may terminate abnormally due to loss of poser, network service interruption, schedule machine downtime, hardware failure, or software errors, such as operating system applications. One of the reasons for the loss of time, waste of computing effort and inability to make time critical decisions is that the applications implementing the model build algorithms assume that a model build will complete successfully. Accordingly, a built model is not saved until the model build data analysis algorithms are completed. Saving model builds only at the completion of model build data analysis algorithms fails to safeguard against the loss of time, waste of computing effort and inability to make time critical decisions in the event of abnormal termination of model builds.
Therefore, there is a need for a checkpointing system and method for long running model build algorithms. In addition, there is a need for the checkpointing to occur through manual, dynamic, and automated initiation. There is also a need for the checkpointing to generate an intermediate representation of a model. Moreover, there is a need for the intermediate representation to be stored in non-volatile storage.